Skip to content

feat(ollama): detect and report GPU vs CPU execution (issue #11)#155

Merged
himanshu231204 merged 1 commit into
mainfrom
fix/ollama-gpu-cpu-detection
Jul 19, 2026
Merged

feat(ollama): detect and report GPU vs CPU execution (issue #11)#155
himanshu231204 merged 1 commit into
mainfrom
fix/ollama-gpu-cpu-detection

Conversation

@himanshu231204

Copy link
Copy Markdown
Member

Summary

Closes #11. When loading a model via Ollama there was no signal whether execution runs on GPU or CPU. This adds a runtime status object that reports the execution device, and surfaces it from the load CLI.

Changes

  • Domain (domain/model.py): added Device enum (gpu / cpu / unknown) and a pure RuntimeStatus pydantic model (backend, available, device, loaded_models, details). No I/O, per the architecture's purity rule.
  • Port (ports/runtime.py): added status() -> RuntimeStatus to RuntimePort.
  • Base runtime (adapters/runtimes/base.py): status() reports availability + device; _detect_device() hook defaults to UNKNOWN so other runtimes get the contract for free.
  • Ollama adapter (adapters/runtimes/ollama.py): _detect_device() calls client.ps() and inspects size_vram per loaded model — > 0 => GPU, 0/None => CPU, no models loaded => UNKNOWN. Best-effort: a failing ps() never breaks the status probe.
  • Manager + CLI (core/manager.py, cli/commands/load.py): ModelManager.runtime_status() delegates to the runtime; load now prints Device: <gpu|cpu|unknown> after a successful load.

Acceptance criteria (from #11)

  • Add a device/executor field to the runtime status object — RuntimeStatus.device
  • Ollama adapter populates it from ollama ps / API metadata — OllamaRuntime._detect_device()
  • CLI load prints the device — cli/commands/load.py
  • Tests with a fake port — unit + port-contract + integration

Tests

  • Unit (tests/unit/test_ollama_runtime.py): GPU (vram>0), CPU (vram=0), UNKNOWN (no models), UNKNOWN (ps raises), unavailable (daemon down) — all with a fake client.
  • Port-contract (tests/unit/test_port_contract.py): status() returns a valid RuntimeStatus with a known device value; FakeRuntime in conftest.py gained status().
  • Integration (tests/integration/test_ollama.py): test_ollama_status_device_detection loads a local model and cross-checks the detected device against ollama ps (auto-skips without a daemon; cloud/subscription models are filtered out).

Verification

Ran against a live Ollama daemon (ollama serve):

  • test_ollama_status_device_detectionpassed; correctly reported device=cpu (machine has no usable GPU; ollama ps showed 100% CPU, size_vram: 0).
  • Unit suite (ollama + port-contract + domain): 43 passed.
  • ruff check clean, mypy --strict clean, ruff format clean.

Notes

  • No core/CLI/API contract changes beyond the additive status() method; existing behavior is unchanged.
  • The device field is best-effort: it reflects currently-loaded models. If nothing is resident, it reports unknown rather than guessing.

🤖 Generated with Claude

Add a RuntimeStatus domain model with a device field (gpu/cpu/unknown) and a status() method on RuntimePort. OllamaRuntime detects the device from ollama ps VRAM usage (size_vram > 0 => GPU). The load CLI now prints the device after a successful load.

Co-Authored-By: Claude <noreply@anthropic.com>
@himanshu231204
himanshu231204 merged commit 09a73b2 into main Jul 19, 2026
29 checks passed
@github-actions

Copy link
Copy Markdown

🎉 Congratulations @himanshu231204!

Your pull request has been successfully merged into main. 🚀

Thank you for contributing to OpenAgentHQ and helping improve the project.

We truly appreciate your contribution and hope to see you back with more amazing PRs!

Happy Open Sourcing! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ollama: detect and report GPU vs CPU execution

1 participant